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LEARNING  CURVE  AND  RATE  ADJUSTMENT  MODELS: 

COMPARATIVE  PREDICTION  ACCURACY  UNDER  VARYING  CONDITIONS 

ABSTRACT 

Learning  curve  models  have  gained  widespread  acceptance  as  a 
technique  for  analyzing  and  forecasting  the  cost  of  items  produced 
from  a  repetitive  process.  Considerable  research  has  investigated 
augmenting  the  traditional  learning  curve  model  with  the  addition 
of  a  production  rate  variable,  creating  a  rate  adjustment  model. 
This  study  compares  the  predictive  accuracy  of  the  learning  curve 
and  rate  adjustment  models.  A  simulation  methodology  is  used  to 
vary  conditions  along  seven  dimensions.  Forecast  errors  are 
analyzed  and  compared  under  the  various  simulated  conditions  using 
ANOVA.  Overall  results  indicate  that  neither  model  dominates;  each 
is  more  accurate  under  some  conditions.  Conditions  under  which 
each  model  tends  to  result  in  lower  forecast  errors  are  identified 
and  discussed. 


LEARNING  CURVE  AND  RATE  ADJUSTMENT  MODELS: 

COMPARATIVE  PREDICTION  ACCURACY  UNDER  VARYING  CONDITIONS 

INTRODUCTION 

Learning  curves  have  gained  widespread  acceptance  as  a  tool  for 
planning,  analyzing,  explaining,  and  predicting  the  behavior  of  the 
unit  cost  of  items  produced  from  a  repetitive  production  process. 
(See  Yelle,  1979,  for  a  review.)  Cost  estimation  techniques  for 
planning  the  cost  of  acquiring  weapon  systems  by  the  Department  of 
Defense,  for  example,  typically  consider  the  role  of  learning  in 
the  estimation  process.  The  premise  of  learning  curve  analysis  is 
that  cumulative  quantity  is  the  primary  driver  of  unit  cost.  Unit 
cost  is  expected  to  decline  as  cumulative  quantity  increases. 

There  is  general  acknowledgement  that  cumulative  quantity  is 
not  the  only  factor  that  influences  unit  cost  and  that  the  simple 
learning  curve  is  not  a  fully  adequate  description  of  cost 
behavior.  Hence  prior  research  has  attempted  to  augment  learning 
curve  models  by  including  additional  variables  (e.g.,  Moses,  1990). 
Most  attention  has  been  focused  on  the  addition  of  a  production 
rate  term.1  The  resulting  augmented  model  is  usually  referred  to 
as  a  rate  adjustment  model. 

Conceptually,  production  rate  should  be  expected  to  affect  unit 
cost  because  of  the  impact  of  economies  of  scale.  Higher 

^ne  review  of  the  literature  pertaining  to  learning  curves 
(Cheney,  1977)  found  that  36%  of  the  articles  reviewed  attempted  to 
augment  the  learning  curve  model  in  some  manner  by  the  inclusion  of 
production  related  variables. 
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production  rates  may  lead  to  several  related  effects:  greater 
specialization  of  labor,  quantity  discounts  and  efficiencies 
associated  with  raw  materials  purchases,  and  greater  use  of 
facilities  permitting  fixed  overhead  costs  to  be  spread  over  a 
larger  output  quantity.  Together,  these  effects  work  to  increase 
efficiency  and  reduce  production  cost  (Bemis,  1981;  Boger  and  Liao, 
1990;  Large,  et.  al.,  1974;  Linder  and  Wilbourn,  1973).  However, 
higher  production  rate  does  not  guarantee  lower  cost.  When 
production  rate  exceeds  capacity,  such  factors  as  over-time  pay, 
lack  of  skilled  labor,  or  the  need  to  bring  more  facilities  online 
may  lead  to  inefficiencies  and  increased  unit  cost.  In  short, 
production  rate  may  be  associated  with  both  economies  and 
diseconomies  of  scale. 

PRIOR  RESEARCH 

Numerous  studies,  using  data  on  actual  production  cost 
elements,  have  been  conducted  to  empirically  examine  the  impact  of 
production  rate  on  unit  cost.  The  broad  objective  of  the  research 
has  been  to  document  rate/cost  relationships  and  determine  if 
consideration  of  production  rate  leads  to  improvements  in  cost 
explanation  or  prediction.  Results  have  been  inconsistent  and 
general  findings  inconclusive.  Various  studies  (e.g. ,  Alchian, 
1963;  Cochran,  1960;  Hirsh,  1952;  Large,  Campbell  and  Cates,  1976) 
found  little  or  no  significance  for  rate  variables.  Other  studies 
did  document  significant  rate/cost  relationships  (e.g.,  Bemis, 
1981;  Cox  and  Gansler,  1981).  Some  research  found  significant 
results  only  for  particular  individual  cost  elements,  such  as  labor 
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(Smith,  1976),  tooling  (Levenson,  et.  al.,  1971)  or  overhead 
(Large,  Hoffmayer,  and  Kontrovich,  1974)  .  But  rate/cost 
relationships  for  these  same ' cost  elements  were  not  consistently 
evident  in  other  studies.  When  significant,  estimates  of  the 
rate/cost  slope  varied  greatly  and  the  direction  of  the 
relationship  was  sometimes  negative  and  sometimes  positive  (e.g., 
Moses,  1990) .  In  reviewing  the  existing  research  on  production 
rate,  Smith  (1980)  concluded  that  a  rate/cost  relationship  may 
exist  but  that  the  existence,  strength  and  nature  of  the 
relationship  varies  with  the  item  produced  and  the  cost  element 
examined. 

Several  explanations  for  these  varying,  inconclusive  empirical 
results  can  be  offered: 

(a)  Varying  results  are  to  be  expected  because  rate  changes  can 
lead  to  both  economies  and  diseconomies  of  scale. 

(b)  Production  rate  effects  are  difficult  to  isolate 
empirically  because  of  colinearity  with  cumulative  quantity 
(Gulledge  and  Womer,  1986) . 

(c)  Researchers  have  usually  used  inappropriate  measures  of 
production  rate  leading  to  misspecified  models  (Boger  and  Liao, 
1990)  . 

(d)  The  impact  of  a  production  rate  change  is  dominated  by 
other  uncertainties  (Large,  Hoffmayer,  and  Kontrovich,  1974) , 
particularly  by  cumulative  quantity  (Asher,  1956) .  Alchian  (1963)  , 
for  example,  was  unable  to  find  results  for  rate  adjustment  models 
that  improved  on  the  traditional  learning  curve  without  a  rate 
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parameter. 


OBJECTIVE  OF  THE  STUDY 

The  prior  research  suggests  that  consideration  of  production 
rate  sometimes  improves  cost  explanation,  but  not  always.  The 
prior  research  suggests  that  a  traditional  learning  curve  model 
sometimes  is  preferable  to  a  rate  adjustment  model,  but  not  always. 
The  prior  research  provides  little  guidance  concerning  the 
circumstances  under  which  explicit  incorporation  of  production  rate 
into  a  learning  curve  model  is  likely  to  lead  to  improved 
explanation  or  prediction.  This  issue  is  important  in  a  number  of 
cost  analysis  and  cost  estimation  situations.  Dorset  (1990)  ,  for 
example,  describes  the  current  situation  facing  military  cost 
estimators  who,  with  the  military  facing  budget  reductions  and 
program  stretchouts,  are  required  to  rapidly  develop  weapon  system 
acquisition  cost  estimates  under  many  different  quantity  profiles. 
One  choice  the  cost  analyst  faces  is  between  using  a  rate 
adjustment  model  or  a  traditional  learning  model  to  develop  estimates.2 

The  objective  of  this  paper  is  to  address  the  following  broad 
issue:  Under  what  circumstances  is  it  beneficial  to  explicitly 


2Two  other  techniques  for  making  cost  estimates  when 
production  rate  changes  are  also  mentioned  by  Dorsett:  curve 
rotation,  which  involves  an  ad  hoc  upward  or  downward  revision  to 
the  slope  of  the  learning  curve,  and  the  use  of  repricing  models 
(e.g.,  Balut,  1981;  Balut,  Gulledge,  and  Womer,  1989)  which  adjust 
learning  curve  estimates  to  reflect  a  greater  or  lesser  application 
of  overhead  cost.  Dorsett  criticized  curve  rotation  for  being 
subjective  and  leading  to  a  compounding  of  error  when  the 
prediction  horizon  is  not  short.  He  criticized  repricing  models 
because  they  must  be  plant-specific  to  be  effective. 
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consider  production  rate  in  cost  analysis?  That  broad  issue  is 
addressed  by  focusing  more  narrowly  on  the  following  research 
questions  Under  what  circumstances  do  rate  adjustment  models 
outperform  traditional  learning  curve  models?  Performance  here 
means  the  ability  to  provide  accurate  predictions  of  future  cost.3 
RESEARCH  APPROACH 

Operationally  the  research  question  implies  a  comparison  of  the 
predictive  accuracy  of  two  competing  cost  estimation  models.  The 
two  competing  models  were  as  follows: 

The  traditional  learning  curve  model,  which  predicts  unit  cost 
as  a  function  of  cumulative  quantity4: 

CL  =  aQb  (1) 

where 

CL  =  Unit  cost  of  item  at  quantity  Q  (i.e.,  with  learning 
considered) . 

Q  =  Cumulative  quantity  produced. 

a  =  Theoretical  first  unit  cost. 

b  =  Learning  curve  exponent  (which  can  be  converted  to  a 
learning  slope  by  slope  =  2b)  . 


3For  many  of  the  studies  cited  in  the  section  on  prior 
research,  performance  was  assessed  in  terms  of  a  model's  ability  to 
ex  post,  statistically  account  for  the  variance  in  an  actual  cost 
series.  The  problem  of  "overf itting"  a  model  to  data  always  exists 
when  regression  techniques  are  used  to  ex  post  explain  a  cost 
series.  In  fact,  if  higher  R2  is  achieved  by  overfitting, 
predictive  ability  can  be  reduced  (Wetherill,  1986). 

4Note  that  this  is  an  incremental  unit  cost  model  rather  than 
a  cumulative  average  cost  model.  Liao  (1988)  discusses  the 
differences  between  the  two  approaches  and  discusses  why  the 
incremental  model  has  become  dominant  in  practice.  One  reason  is 
that  the  cumulative  model  weights  early  observations  more  heavily 
and,  in  effect,  "smooths"  away  period-to-period  changes  in  average 
cost . 
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The  most  widely  used  rate  adjustment  model,  which  modifies  the 
traditional  learning  curve  Model  with  the  addition  of  a  production 
rate  term: 

C„  =  aQ^  (2) 

where 

CR  =  Unit  cost  of  item  at  quantity  Q  and  production  rate  per 
period  R  (i.  e.,  with  production  rate  as  well  as 

learning  considered) . 

Q  =  Cumulative  quantity  produced. 

R  =  Production  rate  per  period  measure. 

a  -  Theoretical  first  unit  cost. 

b  =  Learning  curve  exponent. 

c  -  Production  rate  exponent  (which  can  be  converted  to  a 

production  rate  slope  by  slope  =  2e)  . 

A  simulation  approach  was  used  to  address  the  research 
question.  In  brief,  cost  series  were  generated  under  varying 
simulated  conditions.  The  learning  curve  model  and  the  rate 
adjustment  model  were  separately  fit  to  the  cost  series  to  estimate 
model  parameters.  The  estimated  models  were  then  used  to 
separately  predict  future  cost.  The  relative  accuracy  of  the  two 
alternative  future  cost  predictions  was  determined.  Finally,  an 
analysis  (ANOVA)  was  conducted  relating  relative  prediction 
accuracy  (dependent  variable)  to  the  simulated  conditions 
(independent  variables) . 

There  are  three  main  benefits  gained  from  the  simulation 
approach.  First,  factors  hypothesized  to  influence  prediction 
accuracy  can  be  varied  over  a  wider  range  of  conditions  than  would 
be  encountered  in  any  one  (or  many)  sample (s)  of  actual  cost  data. 
Second,  explicit  control  is  achieved  over  the  manipulation  of 
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factors.  Third,  noise  caused  by  factors  not  explicitly 
investigated  is  removed.  Hence  simulation  provides  the  most 
efficient  way  of  investigating  data  containing  a  wide  variety  of 
combinations  of  the  factor  levels  while  controlling  for  the  effects 
of  other  factors  not  explicitly  identified. 

RESEARCH  CHOICES 

There  were  five  choices  that  had  to  be  made  in  conducting  the 
simulation  experiment: 

(1)  The  form  of  the  rate  adjustment  (RA)  model  whose 
performance  was  to  be  compared  to  the  learning  curve  (LC)  model. 

(2)  The  functional  form  of  the  cost  model  used  to  generate  the 
simulated  cost  data. 

(3)  The  conditions  to  be  varied  across  simulation  treatments. 

(4)  The  cost  objective  (what  cost  was  to  be  predicted) , 

(5)  The  measure  of  prediction  performance. 

Items  (1),  (2),  (4)  and  (5)  deal  with  methodological  issues.  Item 
(3)  deals  with  the  various  conditions  simulated.  The  discussion  of 
item  (3)  encompasses  the  major  variables  examined  in  the  study  and 
hence  the  major  issues  addressed.  Each  item  will  be  discussed  in 
turn. 

1.  The  Rate  Adjustment  Model.  Various  models,  both 
theoretical  and  empirical,  have  been  suggested  for  incorporating 
production  rate  into  the  learning  curve  (Balut  1981;  Balut, 
Gulledge,  and  Womer,  1989;  Linder  and  Wilbourn,  1973;  Smith,  1980, 
1981;  Washburn,  1972;  Womer,  1979).  The  models  vary  with  respect 
to  tradeoffs  made  between  theoretical  completeness  and  empirical 
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tractability .  Equation  2,  described  above,  was  the  specific  rate 
adjustment  model  analyzed  in  this  study,  for  several  reasons: 
First,  it  is  the  most  widely  used  rate  adjustment  model  in  the 
published  literature.  Second,  it  is  commonly  used  today  in  the 
practice  of  cost  analysis  (e.g.,  Dorsett,  1990).  Third,  in 
addition  to  cost  and  quantity  data  (needed  to  estimate  any  LC 
model) ,  equation  2  requires  only  production  rate  data.  Thus 
equation  2  is  particularly  appropriate  for  examining  the 
incremental  benefit  of  attention  to  production  rate.  (Other  RA 
models  offered  in  the  literature  require  knowledge  of  still 
additional  variables) . 5  In  short,  equation  2  is  the  most  widely 
applicable  and  most  generally  used  rate  adjustment '‘model . 

2.  The  Cost  Generating  Function:  A  "true"  cost  function 
for  an  actual  item  depends  on  the  item,  the  firm,  the  time  period 
and  all  the  varying  circumstances  surrounding  actual  production. 
It  is  likely  that  most  manufacturers  do  not  "know"  the  true  cost 
function  underlying  goods  they  manufacture.  Thus  the  choice  of  a 
cost  function  to  generate  simulated  cost  data  is  necessarily  ad 
hoc.  The  objective  here  was  to  choose  a  "generic"  cost  function 
which  had  face  validity,  which  included  components  (parameters  and 
variables)  that  were  generalizable  to  all  production  situations, 
and  which  resulted  in  a  unit  cost  that  depended  on  both  learning 


5The  equation  2  model  is  particularly  applicable  in  situations 
where  a  cost  analyst  or  estimator  does  not  have  ready  access  to  or 
sufficient  knowledge  about  the  cost  structure  and  cost  drivers  of 
a  manufacturer.  Examples  include  the  Department  of  Defense 
procuring  items  from  government  contractors  in  the  private  sector, 
or  prime  contractors  placing  orders  with  subcontractors. 
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and  production  rate  factors.  The  following  explanation  of  the  cost 

function  used  reflects  these  concerns. 

At  the  most  basic  level  the  cost  of  any  unit  is  just  the  sum  of 

the  variable  cost  directly  incurred  in  creating  the  unit  and  the 

share  of  fixed  costs  assigned  to  the  unit,  where  the  amount  of 

fixed  costs  assigned  depend  on  the  number  of  units  produced. 

UC  -  VC  +  £C  (3) 

PQ 

where 

UC  =  Unit  cost. 

VC  =  Variable  cost  per  unit. 

FC  =  Total  fixed  costs  per  period. 

PQ  =  Production  quantity  per  period. 

The  original  concept  of  "learning"  (Wright,  1936)  involved  the 

reduction  in  variable  cost  per  unit  expected  with  increases  in 

cumulative  quantity  produced.  (By  definition,  fixed  costs  are 

assumed  to  be  unaffected  by  volume  or  quantity.)  To  incorporate 

the  effect  of  learning,  variable  cost  can  be  expressed  as: 

VCQ  =  VC^Q*)  (4) 

where 

Q  -  Cumulative  quantity. 

VC0  =  Variable  cost  of  the  Qth  unit. 

VC1  =  Variable  cost  of  the  first  unit, 
d  =  Parameter,  the  learning  index. 

Substituting  into  equation  3: 

UC0  =  VC,(Qd)  +  FC  (5) 

PQ 

Additionally,  assume  the  existence  of  a  "standard"  ("benchmark," 
"normal,"  "planned")  production  quantity  per  period  (PQS)  . 
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Standard  fixed  cost  per  unit  (SFC)  at  the  standard  production 
quantity  would  be: 


SFC  =  £C  (6) 

pQs 

The  production  rate  (PR)  for  any  period  can  then  be  expressed 

as  a  ratio  of  the  production  quantity  to  the  standard  quantity: 

PR  -  PC  (7) 

PQS 

The  second  term  of  equation  (6)  can  then  be  rewritten  as: 

FC  -  SFC  (8) 

PQ  PR 

and  equation  5  rewritten  as: 

UCQ  -  VC1(Qd)  +  SFC  (PR'1)  (9) 


In  this  formulation  it  can  be  seen  that  total  cost  per  unit  is 
the  sum  of  variable  cost  per  unit  (adjusted  for  learning)  plus 
standard  fixed  cost  per  unit  (adjusted  for  production  rate) .  This 
model  incorporates  the  two  factors  presumed  to  impact  unit  costs 
that  have  been  most  extensively  investigated:  cumulative  quantity 
(Q)  and  production  rate  per  period  (PR)  .6  It  is  consistent  with 
both  the  theoretical  and  empirical  literature  which  sees  the 
primary  impact  of  learning  to  be  on  variable  costs  and  the  primary 


6Smith  (1980,  1981) ,  for  example,  used  a  model  similar  to 
equation  9  to  explore  the  effect  of  different  production  rates  on 
unit  cost.  Balut  (1981)  and  Balut,  Gulledge  and  Woroer  (1989) 
construct  models  based  on  learning  and  production  quantity  to 
assist  in  "redistributing"  overhead  and  "repricing"  unit  costs  when 
changes  in  production  rate  occur.  The  Balut  and  Balut,  Gulledge 
and  Womer  models  differ  in  that  they  determine  a  learning  rate  for 
total  (not  variable)  unit  cost  and  then  apply  an  adjustment  factor 
to  allow  for  the  impact  of  varying  production  quantity  on  the 
amount  of  fixed  cost  included  in  total  cost. 
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impact  of  production  rate  to  be  on  the  spreading  of  fixed  costs 
(Smith,  1980) .  Simulated  cost  data  in  this  study  was  generated 
using  equation  9,  while  varying  values  for  the  variables  and 
parameters  on  the  right  hand  side  of  the  equation  to  reflect 
differing  conditions. 

3.  The  Simulated  Conditions;  The  general  research  hypothesis 
is  that  the  relative  predictive  accuracy  of  the  LC  and  RA  models 
will  depend  on  the  circumstances  in  which  they  are  used.  What 
conditions  might  be  hypothesized  to  affect  prediction  accuracy? 
Seven  different  factors  (independent  variables)  were  varied  during 
the  simulation.  In  the  following  paragraphs,  each  factor  is 
discussed.  A  label  for  each  is  provided,  along  with  a  discussion 
of  why  the  factor  may  be  relevant  to  prediction  accuracy  and  how 
the  factor  was  operationalized  in  the  simulation.  Table  1 
summarizes  the  seven  factors. 

i)  Data  History  (DATAHIST) :  The  number  of  data  points 
available  to  estimate  parameters  for  a  model  should  affect  the 
accuracy  of  a  model.  More  data  available  during  the  "estimation 
period"  should  be  associated  with  greater  accuracy  for  both  the  LC 
and  the  RA  model.7  But  increased  data  may  not  impact  the  accuracy 
of  the  two  models  in  the  same  manner.  The  RA  model  requires  the 
estimation  of  an  additional  parameter,  and  consequently  is  likely 
to  require  more  data  points  to  avoid  unreliable  parameter  estimates 
and  resultant  prediction  inaccuracy.  Hence  it  may  be  hypothesized 

rThere  are,  of  course,  cost/benefit  tradeoffs.  The  marginal 
benefits  of  increased  prediction  accuracy  for  any  model  must  be 
weighed  against  the  marginal  costs  of  additional  data  collection. 
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that  the  RA  model  will  be  more  sensitive  (less  accurate)  when  data 
is  lean. 

In  the  simulation,  data  history  was  varied  from  four  to  seven 
to  ten  data  points  available  for  estimating  model  parameters.  This 
simulates  having  knowledge  of  costs  and  quantities  for  four,  seven 
or  ten  production  lots.  Four  is  the  minimum  number  of  observations 
needed  to  estimate  the  parameters  of  the  RA  model  by  regression. 
The  simulation  focuses  on  lean  data  availability  both  because  the 
effects  of  marginal  changes  in  data  availability  should  be  most 
pronounced  when  few  observations  are  available  and  because  many 
real  world  applications  (e.g.,  cost  analysis  of  Department  of 
Defense  weapon  system  procurement)  occur  under  lean  data 
conditions. 

ii)  Variable  Cost  Learning  Rate  (LEARNRAT)  :  In  the  cost 
generating  function,  learning  affects  total  unit  cost  by  affecting 
variable  cost  per  unit.  Past  research  (Smunt,  1986)  has  shown  that 
the  improvement  in  prediction  accuracy  from  including  a  learning 
parameter  in  a  model  (when  compared  to  its  absence)  depends  on  the 
degree  of  learning  that  exists  in  the  underlying  phenomena  being 
modeled;  the  greater  the  degree  of  learning,  the  more  benefit 
gained  from  inclusion  of  a  learning  rate  term.  This  suggests  that 
prediction  accuracy  may  depend  on  the  learning  rate  and  hence  that 
relative  prediction  accuracy  of  the  LC  and  RA  models  may  also 
depend  on  the  learning  rate.  In  the  simulation,  variable  cost 
learning  rate  (reflected  in  parameter  d  in  equation  9)  was  varied 
from  75%  to  85%  to  95%.  Generally,  complex  products  or  labor 


12 


TABLE  1 


INDEPENDANT  VARIABLES 


Concent 

Label 

Levels 

Data  History 

DATAHIST1 

4 

7 

10 

Variable  Cost  Learning 
Rate 

LEARNRAT 

75% 

85% 

95% 

Fixed  Cost  Burden 

BURDEN2 

15% 

33% 

50% 

Production  Rate  Trend 

PROTREND3 

Level 

Growth 

Production  Rate 
Instability/Variance 

RATEVAR4 

.05 

.  15 

.25 

Cost  Noise/Variance 

COSTVAR5 

.05 

.  15 

.25 

Future  Production  Level 

FUTUPROD6 

Low 

Same 

High 

’Number  of  data  points  available  during  the  model  estimation 
period?  simulates  the  number  of  past  production  lots. 

Standard  per  unit  fixed  cost  as  a  percentage  of  cumulative 
average  per  unit  total  cost,  during  the  model  estimation  period. 

3A  level  trend  means  production  at  100%  of  standard  production 
for  each  lot  during  the  estimation  period.  A  growth  trend  means 
production  rate  gradually  increasing  to  100%  of  standard  production 
during  the  estimation  period.  The  specific  growth  pattern  depends 
on  the  number  of  production  lots  in  the  estimation  period,  with 
sequences  as  follows  (expressed  as  a  %  of  standard) :  For  DATAHIST 
=  4:  33%,  67%,  100%,  100%.  For  DATAHIST  -  7:  20%,  40%,  60%,  80%, 
100%,  100%,  100%.  For  DATAHIST  -  10?  10%,  20%,  35%,  50%,  70%,  90%, 
100%,  100%,  100%,  100%. 

Coefficient  of  variation  of  production  rate.  (Degree  of 
instability  of  production  rate  around  the  general  production  rate 
trend. ) 

Coefficient  of  variation  of  total  per  unit  cost. 

s"Sarae"  means  production  rate  at  100%  of  standard  for  each 
lot  produced  within  the  prediction  zone.  "Low"  means  production 
rate  at  50%.  "High"  means  production  rate  at  150%. 


total  unit  cost  made  up  of  fixed  cost.9  Three  percentages  were  used 
in  the  simulation:  15%,  33%,  and  50%.  The  different  percentages 
can  be  viewed  as  simulating  different  degrees  of  operating 
leverage,  of  capital  intensiveness,  or  of  plant  automation.  The 
15%  level  reflects  the  average  fraction  of  price  represented  by 
fixed  overhead  in  the  aerospace  industry,  as  estimated  at  one  time 
by  DOD  (Balut,  1981). 10  The  larger  percentages  are  consistent  with 
the  trend  toward  increased  automation  (McCullough  and  Balut,  1986) . 

iv)  Production  Rate  Trend  (PROTREND) :  When  initiating  a  new 
product,  it  is  not  uncommon  for  the  production  rate  per  period  to 
start  low  and  trend  upward  to  some  "normal"  level.  This  may  be  due 
both  to  the  need  to  develop  demand  for  the  output  c~  the  desire  to 
start  with  a  small  production  volume,  allowing  slack  for  working 
ougs  out  of  the  production  process.  Alternatively,  when  a  "new" 
product  results  from  a  relatively  small  modification  of  an  existing 
product,  sufficient  customer  demand  or  sufficient  coi  fidence  in  the 


’operationally  this  is  a  bit  complex,  since  both  per  unit 
variable  and  per  unit  fixed  cost  depend  on  other  simulation  inputs 
(cumulative  quantity  and  production  rate  per  period) .  The  process 
of  relating  fixed  cost  to  total  cost  was  as  follows:  First,  a 
cumulative  average  per  unit  variable  cost  for  all  units  produced 
during  the  estimation  period  was  determined.  Then  a  standard  fixed 
cost  per  unit  was  set  relative  to  the  cumulative  average  per  unit 
variable  cost.  For  example,  if  standard  fixed  cost  per  unit  was 
set  equal  to  cumulative  average  variable  cost  per  unit,  then  "on 
average"  fixed  cost  would  comprise  50%  of  total  unit  cost  during 
the  estimation  period.  Actual  fixed  cost  per  unit  may  differ  from 
standard  fixed  cost  per  unit  if  the  production  rate  (discussed 
later)  was  not  at  100%  of  standard. 

10In  the  absence  of  firm-specific  cost  data,  the  Cost  Analysis 
Improvement  Group  in  the  Office  of  the  Secretary  of  Defense  treats 
15%  of  the  unit  price  of  a  defense  system  as  representing  fixed 
cost  (Pilling,  1990) . 
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production  process  may  be  assumed  and  full  scale  production  may  be 
initiated  rapidly.  In  short,  two  different  patterns  in  production 
volume  may  be  exhibited  early  on  when  introducing  a  new  item:  a 
gradual  growing  trend  toward  full  scale  production  or  a  level  trend 
due  to  introduction  at  full  scale  production  volume. 

How  might  the  production  rate  trend  during  the  estimation 
period  affect  the  relative  prediction  accuracy  of  the  LC  and  the  RA 
models?  Two  contrary  arguments  seem  relevant.  On  the  one  hand, 
the  purpose  of  an  RA  model  is  to  explain  variance  in  cost  due  to 
variance  in  production  rate.  If  production  rate  is  virtually  level 
during  a  model  estimation  period,  there  would  be  little  period-to- 
period  differences  in  unit  cost  caused  by  the  spreading  of  fixed 
costs  over  varying  outputs.  Any  rate  effects  would  likely  be 
swamped  by  other  random  impacts  on  cost.  This  suggests  greater 
incremental  benefit  to  using  an  RA  model  when  the  production  rate 
has  not  been  level,  i.e.,  when  growth  occurred  during  the  model 
estimation  period.  On  the  other  hand,  a  growing  production  rate 
from  period  to  period  results  in  statistical  problems.  Within  the 
RA  model,  there  will  be  greater  colinearity  between  cumulative 
quantity  (necessarily  growing  each  period)  and  production  rate. 
Empirically,  this  colinearity  has  been  observed  by  many  cost 
analysts  (e.  g.,  Gulledge  and  Womer,  1986).  The  coljnearity  makes 
production  rate  a  somewhat  redundant  variable  and  causes  unreliable 
parameter  estimates.  This  suggests  less  incremental  benefit  to 
using  an  RA  model  if  production  rate  has  been  growing. 

Which  argument  holds  is  an  empirical  question.  The  simulation 
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created  two  production  trends  during  the  model  estimation  period: 
••level*'  and  "growth."  These  represented  general  trends  (but,  as 
will  become  clear  momentarily,  variance  around  the  general  trend 
was  introduced) .  The  level  trend  simulated  a  production  rate  set 
at  a  "standard"  100%  each  period  during  model  estimation.  The 
growth  trend  simulated  production  rate  climbing  gradually  to  100%. 
Details  of  the  trends  are  in  table  1. 

v)  Production  Rate  Instability /Variance  (RATEVAR) :  Numerous 
factors,  in  addition  to  the  general  trend  in  output  discussed 
above,  may  operate  to  cause  period-to-period  fluctuations  in 
production  rate.  Manufacturers  typically  do  not  have  complete 
control  over  either  demand  for  output  or  supply  of  inputs. 
Conditions  in  either  market  can  cause  instability  in  production 
rate.  (Of  course,  unstable  demand,  due  to  the  uncertainties  of 
annual  budget  negotiations,  is  claimed  to  be  a  major  cause  of  cost 
growth  during  the  acquisition  of  major  weapon  systems  by  the  DoD) . 
The  concern  here  is  with  the  likely  effect  of  rate  instability  on 
the  relative  accuracy  of  LC  and  RA  models.  If  production  rate  is 
highly  stable  across  periods,  there  would  be  little  variance  in 
cost  outcomes  on  which  to  estimate  a  rate  parameter.  Rate  effects 
would  be  dominated  by  other  impacts  on  cost.  This  suggests  greater 
benefit  to  using  an  RA  model  when  production  rate  is  unstable  over 
time. 

Production  rate  instability  was  simulated  by  adding  random 
variance  to  each  period's  production  rate  during  the  estimation 
period.  The  amount  of  variance  ranged  from  a  coefficient  of 
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variation  of  .05  to  .15  to  .25.  For  example,  if  the  production 
trend  was  level  and  the  coefficient  of  variation  was  .05  then 
••actual”  production  rates  simulated  were  generated  by  a  normal 
distribution  with  mean  equal  to  the  standard  production  rate  (100%) 
and  sigma  equal  to  5%. 

vi)  Cost  Noise/Variance  (COSTVAR) :  From  period  to  period 
there  will  be  unsystematic,  unanticipated,  non-recurring,  random 
factors  that  will  impact  unit  cost.  Changes  in  the  cost,  type  or 
availability  of  input  resources,  temporary  increases  or  decreases 
in  efficiency,  and  unplanned  changes  in  the  production  process  are 
all  possible  causes.  Conceptually  such  unsystematic  factors  can  be 
thought  of  as  adding  random  noise  to  unit  cost.  While  unsystematic 
variation  in  cost  cannot  (by  definition)  be  controlled,  it  is  often 
possible  to  characterize  different  production  processes  in  terms  of 
the  degree  of  unsystematic  variation?  some  processes  are  simply 
less  well -understood,  more  uncertain,  and  less  stable  than  others. 

Does  the  relative  predictive  accuracy  of  LC  and  RA  models 
depend  on  the  stability  of  the  process  underlying  cost?  To 
investigate  this  question,  random  variance  was  added  to  the 
simulated  costs  generated  from  the  cost  function.  The  amount  of 
variance  ranged  from  a  coefficient  of  variation  of  .05  to  .15  to 
.25.  For  example,  when  the  coefficient  of  variation  was  .25,  then 
"actual''  unit  costs  simulated  were  generated  by  a  normal 
distribution  with  mean  equal  to  cost  from  equation  9  and  sigma 
equal  to  25%. 

vii)  Future  Production  Level  (FUTUPROD) :  Once  a  model  is 
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constructed  (from  data  available  during  the  estimation  period) ,  it 
is  to  be  used  to  predict  future  cost.  The  production  rate  planned 
for  the  future  may  vary  from  past  levels.  Further  growth  may  be 
planned.  Cutbacks  may  be  anticipated.  Will  the  level  of  the 
future  production  rate  affect  the  relative  predictive  accuracy  of 
the  LC  and  RA  models?  Is  one  model  more  accurate  if  cutbacks  in 
production  are  anticipated  and  another  if  growth  is  planned? 
Conflicting  scenarios  can  be  argued.  For  example,  on  the  one  hand, 
inclusion  of  a  rate  term  might  be  expected  to  increase  prediction 
accuracy  when  production  rate  changes  significantly  (i.  e.,  either 
growth  or  decline  in  the  future  period).  On  the  other  hand,  if 

l 

growth  during  the  future  prediction  period  merely  continues  a 
growth  trend  established  during  the  estimation  period,  then 
production  rate  will  once  again  tend  to  be  correlated  with 
cumulative  quantity,  and  the  incremental  benefit  of  an  RA  model 
over  an  LC  model  may  be  less  obvious. 

In  the  simulation,  future  production  was  set  at  three  levels: 
low  (50%  of  standard),  same  (100%  of  standard)  and  high  (150%  of 
standard) .  These  simulate  conditions  of  cutting  back,  maintaining 
or  increasing  production  relative  to  the  level  of  production 
existing  at  the  end  of  the  model  estimation  period. 

4.  The  Cost  Objective:  What  is  to  be  predicted?  Op  to  this 
point  the  stated  purpose  of  the  study  has  been  to  evaluate  accuracy 
when  predicting  future  cost.  But  which  future  cost?  Three 
alternatives  were  examined. 

i)  Next  period  average  unit  cost:  As  the  label  suggests  this 
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is  the  average  per  unit  cost  of  items  in  the  production  "lot" 
manufactured  in  the  first  period  following  the  estimation  period. 
Here  the  total  cost  of  producing  the  output  for  the  period  is 
simply  divided  by  the  output  volume  to  arrive  at  unit  cost. 
Attention  to  this  cost  objective  simulates  the  need  to  predict  near 
term  unit  cost. 

ii)  Total  cost  over  a  finite  production  horizon:  The 
objective  here  is  to  predict  the  total  cost  of  all  units  produced 
during  a  fixed  length  production  horizon.  Three  periods  were  used 
as  the  length  of  the  production  horizon  (one  production  lot 
produced  each  period) .  If  the  future  production  rate  is  low  (hiyh) 
then  relatively  few  (many)  units  will  be  produced  during  the  finite 
production  horizon.  Attention  to  this  cost  objective  simulates  the 
need  to  predict  costs  over  some  specific  planning  period, 
regardless  of  the  volume  to  be  produced  during  that  planning 
period. 

iii)  Total  program  cost:  The  objective  here  is  to  predictive 
total  cost  for  a  specified  number  of  units.  If  the  future 
production  rate  is  low  (high)  then  relatively  more  (fewer)  periods 
will  be  required  to  manufacture  the  desired  output.  The  simulation 
was  constructed  such  that  at  a  low  (same,  high)  level  of  future 
production  six  (three,  two)  future  periods  were  required  to  produce 
the  output.  Attention  to  this  cost  objective  simulates  the  need  to 
predict  total  cost  for  a  particular  production  program,  regardless 
of  the  number  of  future  periods  necessary  to  complete  the  program. 

Examining  each  of  these  three  cost  objectives  was  deemed 
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necessary  to  provide  a  well-rounded  investigation  of  predictive 
accuracy.  However,  the  findings  were  substantially  the  same  across 
the  three  cost  objectives.  In  the  interests  of  space,  the 
remainder  of  this  paper  will  discuss  the  analysis  and  results  only 
for  the  first  cost  objective,  the  average  cost  per  unit  for  the 
next  period's  output. 

5.  The  Measures  of  Prediction  Performance;  In  order  to 
compare  prediction  accuracy  between  LC  and  RA  models,  two  kinds  of 
measures  were  required.  First,  a  model  prediction  error  (ERR)  was 
determined  separately  for  each  (LC  or  RA)  model  as  follows: 

ERR  =  |PUC  -  AUC  |  4-  AUC 

where  \ 

PUC  -  Predicted  unit  cost  from  either  the  learning  curve 
or  the  rate  adjustment  model. 

AUC  =  Actual  unit  cost  as  generated  by  the  cost 
function. 

This  is  a  commonly  used  error  measure,  the  absolute  percentage 
error. 

Next  a  simple  difference  in  errors  (ERRDIFF)  between  the  LC 
prediction  and  the  RA  prediction  was  calculated. 

ERRDIFF  =  ERRlc  -  ERRra 

where 


ERR 


LC 


Absolute  percentage  error  using  the  learning  curve 
model  prediction  of  cost. 


ERRra  =  Absolute  percentage  error  using  the  rate 
adjustment  model  prediction  of  cost. 

Positive  values  for  ERRDIFF  mean  that  the  rate  adjustment  model 

produced  smaller  prediction  errors  that  the  learning  curve  model 
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and  consequently  imply  increased  accuracy  from  incorporating  a 
production  rate  variable  into  the  prediction  process.  Negative 
values  mean  the  learning  curve  model  is  more  accurate.  ERRDIFF 
represents  the  dependent  variable  in  the  statistical  analysis.  The 
research  question  then  becomes:  What  factors  or  conditions  explain 
variance  in  ERRDIFF? 

Figure  1  summarizes  the  complete  simulation  process  leading  up 
to  the  determination  of  ERRDIFF.  The  simulation  was  run  once  for 
each  possible  combination  of  treatments.  Given  seven  factors 
varied  and  three  possible  values  for  each  factor  (except  for 
PROTREND  which  had  two) ,  there  were  3x3x3x3x3x3x2  = 
1458  combinations.  Thus  the  simulation  generated  1458  observations 
and  1458  values  for  ERRDIFF.11 


ANALYSIS  AND  FINDINGS 

The  simulation  results  were  evaluated  using  analysis  of 
variance  (ANOVA)  to  conduct  tests  of  statistical  significance.  All 
main  effects  are  discussed  in  the  following  section.  First  order 
(pairwise)  interactions  were  also  tested.  Those  significant  at  .01 


^In  the  simulation,  just  as  in  the  real  practice  of  cost 
analysis,  it  is  possible  for  a  model  estimated  on  limited  data  to 
be  very  inaccurate,  leading  to  extreme  values  for  ERR  (or  ERRDIFF). 
If  such  outlier  values  were  to  be  used  in  the  subsequent  analysis, 
findings  would  be  driven  by  the  outliers.  Screening  of  the 
observations  for  outliers  was  necessary.  During  the  simulation,  if 
a  model  produced  an  ERR  value  in  excess  of  100%,  then  that  value 
was  replaced  with  100%.  This  truncation  has  the  effect  of  reducing 
the  impact  of  an  outlier  on  the  analysis  while  still  retaining  the 
observation  as  one  that  exhibited  poor  accuracy.  Alternative 
approaches  to  the  outlier  problem  included  deletion  instead  of 
truncation  and  use  of  a  50%  ERR  cutoff  rather  than  the  100%  cutoff. 
Findings  were  not  sensitive  to  these  alternatives. 
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FIGURE  1 
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are  also  discussed.  Table  2  provides  the  ANOVA  results. 


Main  Effects:  As  indicated  in  table  2,  all  main  effects, 
except  LEARNRAT,  are  significant  at  .0005,  indicating  that  values 
for  ERRDIFF  are  significantly  influenced  by  the  treatment 
conditions.  Table  3  summarizes  ERRDIFF  values  under  the  various 
experimental  conditions. 

The  major  findings  are  perhaps  made  most  obvious  by  a  plot  of 
ERRDIFF  by  treatments.  Figure  2  shows  a  sample  plot  of  ERRDIFF 
against  DATAHIST.  Two  points  are  of  interest.  First,  the  relative 
accuracy  of  the  LC  and  RA  models  clearly  does  depend  on  the  number 
of  observations  available  for  model  estimation.  Second,  both 
negative  and  positive  value  for  ERRDIFF  are  present'.  This  implies 
that  under  some  conditions  (few  observations)  the  LC  model  is  more 
accurate  than  the  RA  model.  But  under  different  conditions  (larger 
number  of  observations)  the  RA  model  is  more  accurate.  The 
crossover  point  occurs  at  about  seven  observations. 

Figure  3  shows  a  similar  plot  of  ERRDIFF  with  all  (significant) 
variables  superimposed.  In  this  plot,  1,  2,  and  3  on  the  x-axis 
reflect  low,  medium,  and  high  values  for  the  independent  variables 
(which  are  taken  from  the  left,  middle  and  right  columns  of  Table 
3)  .  Figure  3  makes  several  points.  First  for  each  the  six 
variables  plotted,  ERRDIFF  varies  significantly  depending  on  the 
value  (treatment)  for  the  independent  variable.  This  simply 
implies  that  the  experimental  conditions  do  reflect  factors 
relevant  to  model  accuracy. 

Second  for  all  six  independent  variables,  at  some  point  over 
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TABLE  2 


ANALYSIS  OF  VARIANCE  RESULTS 


SOURCE 

£E 

SUM  OF  SQUARES 

MEAN  SQUARE 

F  VALUE 

Model 

85 

33.10921 

.3895 

9.03 

Error 

1372 

59.18006 

.0431 

PR>F: 

Corrected  Total 

1457 

92.28927 

.0001 

E? _ 

£V _ 

ERRDIEF  MEAN 

.3588 

1236.81 

-.0168 

SOURCE 

££ 

ANOVA  SS 

F  VALUE 

PR>F 

DATAHIST 

2 

3.6780 

42.63 

.  0001 

LEARN RAT 

2 

.0510 

.59 

.  5536 

BURDEN 

2 

.7383 

8.56 

.  0002 

PROTREND 

1 

1.6259 

37.70 

.  0001 

RATEVAR 

2 

5.4728 

63.44 

.0001 

COSTVAR 

2 

9.3918 

108.87 

.  0001 

FUTUPROD 

2 

.6662 

7.72 

.0005 

DATAHIST* LEARN RAT 

4 

.0699 

.41 

.  8050 

DATAHIST*BURDEN 

4 

.1772 

1.03 

.  3920 

DAT AH 1ST*  PROTREND 

2 

.  32.17 

3.73 

.  0243 

DATAHIST*RATEVAR 

4 

.0844 

.49 

.7436 

DATAHIST*COSTVAR 

4 

.9813 

5.69 

.  0002 

DAT AH IST*FUTUPROD 

4 

.7531 

4.37 

.0016 

LEARNRAT*BURDEN 

4 

.2444 

1.42 

.  2260 

LE ARNRAT  *  PROTRE  N  D 

2 

.0092 

.  11 

.8979 

LEARNRAT*RATEVAR 

4 

.3933 

2 . 28 

.0588 

LE ARNRAT * COSTVAR 

4 

.2407 

1.40 

.2332 

LEARNRAT*FUTUPROD 

4 

.3818 

2.21 

.  0655 

BURDEN* PROTREND 

2 

.0806 

.93 

.3931 

BURDEN*RATEVAR 

4 

.2325 

1.35 

.2501 

BURDEN*COSTVAR 

4 

.7304 

4.23 

.0021 

BURDEN*FUTUPROD 

4 

1.0539 

6.11 

.0001 

PROTREN  D  *  RATEVAR 

2 

2.1867 

25.35 

.0001 

PROTREN D * COS TV AR 

2 

.1022 

1.19 

.3059 

PROTREND* FUTUPROD 

2 

.4751 

5.51 

.0041 

RATEVAR* COSTVAR 

4 

.3163 

1.83 

.  1199 

RATE VAR * FUTU PRO D 

4 

1.7734 

10.28 

.0001 

COSTVAR* FUTUPROD 

4 

.8760 

5.08 

.0005 

TABLE  3 


PREDICTION  ERROR  DIFFERENCE  BETWEEN 
LEARNING  CURVE  AND  RATE  ADJUSTMENT  MODELS: 
MAIN  EFFECTS 


DATAHIST  Value: 

4 

7 

10 

ERRDIFF  Mean: 

-.87 

.007 

.030 

LEARNRAT  Value: 

75% 

85% 

95% 

ERRDIFF  Mean: 

-.009 

-.019 

-.023 

BURDEN  Value: 

15% 

33% 

50% 

ERRDIFF  Mean: 

-.047 

-.009 

.006 

PROTREND  Value: 

level 

- 

growth 

ERRDIFF  Mean: 

-.050 

\ 

.017 

RATEVAR  Value: 

.05 

.15 

.25 

ERRDIFF  Mean: 

-.095 

-.010 

.054 

COSTVAR  Value: 

.05 

.15 

.25 

ERRDIFF  Mean: 

.082 

-.019 

-.114 

FUTUPROD  Value 

low 

same 

high 

ERRDIFF  Mean: 

.  003 

-.007 

-.046 

FIGURE  3 


the  range  of  these  variables,  both  positive  and  negative  values  for 
ERRDIFF  exist.  This  means  that  neither  the  LC  model  nor  the  RA 
model  dominates.  Each  model  is  more  accurate  under  some 
experimental  conditions. 

Third,  some  factors  apparently  have  a  greater  impact  on 
relative  prediction  accuracy  than  others.  For  example,  the  plot 
shows  that  both  the  most  extreme  values  for  ERRDIFF  (highest 
positive  and  lowest  negative)  are  associated  with  different 
treatments  on  COSTVAR.  The  F-value  for  COSTVAR  is  also  the  highest 
and  most  significant  in  the  ANOVA.  This  suggests  that  the  greatest 
impact  on  relative  predictive  accuracy  is  due  to  variation  in  the 
amount  of  unsystematic  noise  impacting  the  measure  of  cost.  A 
relatively  large  effect  is  also  apparent  due  to  variation  in 
RATEVAR;  while  relatively  small  (but  still  significant)  effects  are 
associated  with  variations  in  BURDEN  and  FUTUPROD.  These 
observations  are  interesting  but  must  be  interpreted  with  caution. 
COSTVAR  does  have  the  greatest  impact  on  ERRDIFF  over  the 
particular  range  of  treatments  examined  by  the  simulation.  But  if 
the  ranges  over  which  the  individual  independent  variables  were 
allowed  to  vary  were  altered,  a  different  factor  could  appear  to  be 
most  important. 

Fourth,  values  for  ERRDIFF  are  either  monotonically  increasing 
or  decreasing  for  the  various  independent  variable  treatments. 
Hence  some  general  conclusions  about  the  impact  of  the  experimental 
conditions  on  relative  prediction  errors  are  possible: 

a)  Data  History:  As  indicated  above  the  RA  model  outperforms 
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the  LC  model  when  the  number  of  observations  for  model  estimation 
is  relatively  greater.  This  is  expected;  the  benefit  from  greater 
precision  due  to  the  additional  term  in  the  RA  model  can  only  be 
realized  when  the  data  are  sufficient  to  produce  reliable  parameter 
estimates . 

b)  Fixed  Cost  Burden:  As  expected  the  RA  model  tends  to 
outperform  the  LC  model  as  the  proportion  of  total  unit  cost 
comprised  of  fixed  cost  increases.  This  implies  that  the  benefit 
derived  from  use  of  a  rate  adjustment  model  depends  on  the  capital 
intensiveness  of  the  production  process.  Surprising,  perhaps,  was 

the  fact  that  the  change  in  ERRDIFF  from  the  low  (15%)  fixed  cost 

\ 

treatment  to  the  high  (50%)  was  relatively  small. 

c)  Past  Production  Trend:  The  RA  model  tends  to  outperform 
the  LC  model  if  the  production  rate  was  growing  during  the  model 
estimation  period.  This  finding  suggests  that  it  is  necessary  to 
have  a  changing  production  rate  from  period-to-period  in  order  for 
the  model  estimation  process  to  reliably  capture  the  effect  of  rate 
on  cost.  It  also  suggests  that  relatively  higher  correlation 
between  cumulative  quantity  and  production  rate  (which  will  occur 
when  rate  has  been  growing)  does  not  imply  that  cumulative  quantity 
will  be  a  sufficient  variable.  Cumulative  quantity  does  not 
adequately  capture  the  impact  of  rate  changes. 

d)  Production  Rate  Instability:  The  RA  model  tends  to 
outperform  the  LC  model  when  there  is  greater  instability  in 
production  rate.  This  suggests  that  it  is  desirable  to  have  large 
period-to-period  variation  in  production  rate  during  the  time  of 
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model  estimation.  In  retrospect  this  conclusion  seems  obvious.  If 
the  variance  in  cost  caused  by  rate  changes  is  small,  because  the 
rate  changes  themselves  are  small,  then  the  effect  will  be  swamped 
by  other  effects  and  model  parameters  will  be  unreliable. 

e)  Cost  Noise/Variance:  As  the  random  noise  in  unit  cost 
increases,  the  RA  model  will  perform  more  poorly  relative  to  the  LC 
model.  This  suggests  that  when  there  are  unsystematic, 
unpredictable  factors  that  influence  cost,  use  of  the  RA  model  is 
less  beneficial.  The  noise  leads  to  unreliable  estimates  of  the 
production  rate  effect  and  poorer  cost  predictions. 

f)  Future  Production  Level:  The  degree  to  which  the  RA  model 
outperforms  the  LC  model  appears  to  be  inversely  associated  with 
the  production  rate  in  the  prediction  period.  When  cutbacks  in 
production  are  anticipated,  cost  is  more  accurately  predicted  by 
the  RA  model.  When  growth  is  anticipated,  cost  is  more  accurately 
predicted  by  the  LC  model.  This  suggests  that  the  RA  model  is  most 
beneficial  when  a  reversing  of  the  production  trend  from  the 
estimation  period  to  the  prediction  period  is  expected. 

Interaction  Effects:  Eight  first  order  interactions  were  also 
found  to  be  significant  in  the  ANOVA.  Note  that  five  of  these 
interactions  involve  FUTUPROD  as  one  of  the  interacting  variables. 
This  means  that  the  impact  of  other  variables  on  relative 
prediction  accuracy  is  particularly  sensitive  to  the  anticipated 
level  of  production  in  the  prediction  period.  Examining  these 
eight  interactions  individually  leads  to  further  insight  concerning 
the  conditions  that  impact  relative  prediction  accuracy.  They  are 
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illustrated  in  Figures  4  through  11. 

Figure  4,  the  interaction  of  Data  History  with  Future 
Production  Level,  demonstrates  again  that  the  RA  model  tends  to 
outperform  the  LC  model  as  the  data  history  becomes  richer.  But 
this  effect  is  most  pronounced  when  a  low  future  production  level 
is  planned.  If  a  cutback  in  production  is  projected  and  few 
observations  are  available  for  model  estimation,  the  LC  model  is 
substantially  more  accurate  than  the  RA  model  (the  most  negative 
value  for  ERRDIFF  occurs  under  these  conditions) .  If  a  cutback  in 
production  is  projected  and  relatively  many  observations  are 
available  for  model  estimation,  the  RA  model  is  substantially  more 
accurate  than  the  LC  model.  (The  most  positive  valve  for  ERRDIFF 
occurs  under  these  conditions.) 

Figure  5,  the  interaction  of  Fixed  Cost  Burden  and  Future 
Production  Level,  demonstrates  a  somewhat  analogous  effect.  The  RA 
model  tends  toward  better  accuracy  than  the  LC  model  as  fixed  cost 
burden  increases.  But  again  this  effect  is  most  pronounced  when 
low  levels  of  production  in  the  future  are  planned.  The  analyst's 
concern  for  the  impact  of  burden  percentage  on  model  accuracy 
should  be  greatest  when  volume  cutbacks  are  expected. 

Figure  6,  the  interaction  of  Production  Rate  Instability  and 
Future  Production  Level  also  shows  the  importance  of  the  future 
production  level.  The  RA  model  tends  to  outperform  the  LC  model  as 
the  degree  of  variability  in  the  production  rate  increases.  But 
the  impact  of  rate  variability  is  most  pronounced  when  a  cutback  in 
production  volume  is  planned.  If  the  production  rate  was  stable 
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FIGURE  4 


INTERACTION  OF  DATA  HISTORY 
WITH  FUTURE  PRODUCTION  LEVEL 
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FIGURE  7 


during  the  model  estimation  period,  the  LC  model  is  more  accurate. 
If  the  production  rate  was  fluctuating  and  a  cutback  in  volume  is 
planned,  then  the  RA  model  substantially  outperforms  the  LC  model. 

Figure  7 ,  the  interaction  of  Cost  Noise  with  Future  Production 
Level,  again  illustrates  how  the  future  production  volume  impact* 
relative  prediction  accuracy.  For  all  treatments,  as  unsystematic 
factors  make  cost  more  noisy,  the  RA  model  tends  to  lose  its 
relative  advantage  over  the  simpler  LC  model.  But  this 
deterioration  in  predictive  accuracy  due  to  noise  for  the  RA  model 
is  more  evident  when  a  cutback  in  future  production  volume  is 
anticipated. 

Figure  8  shows  the  interaction  of  the  Production  Trend  during 
the  estimation  period  and  the  Future  Production  Level  during  the 
prediction  period.  Again  the  steepest  curve  occurs  when  the  future 
production  level  is  low.  This  plot  demonstrates  that  if  production 
rate  is  relatively  level  during  the  model  estimation  period,  the 
simple  LC  model  tends  to  outperform  the  RA  model.  But  the  RA  model 
outperforms  the  LC  model  when  there  is  a  shift  or  change  in 
direction  of  the  production  trend  from  the  estimation  period  to  the 
prediction  period.  This  follows  from  the  positive  value  for 
ERRDIFF  when  a  growing  past  production  trend  is  coupled  with  a 
decline  in  future  production  level. 

Figure  9  shows  an  interesting  interaction  between  Cost  Noise 
and  Fixed  Cost  Burden.  The  plot  shows  that  under  all  conditions, 
increasing  cost  noise  reduces,  and  then  eliminates,  any  benefit 
from  using  the  RA  model.  But  the  deterioration  is  most  dramatic  if 


27 


the  percentage  of  fixed  cost  in  total  cost  is  high.  If  burden  is 
high  and  cost  is  not  subject  to  unsystematic  noise ,  then  the  RA 
model  is  substantially  superior.  But  if  burden  is  high  and  cost 
is  subject  to  unsystematic  noise,  the  RA  model  is  substantially 
inferior. 

Figure  10,  the  interaction  between  Cost  Noise  and  Data  History, 
illustrates  a  somewhat  additive  effect  of  two  factors.  Either 
increasing  cost  noise  or  reducing  the  number  of  observations  causes 
deterioration  in  the  performance  of  the  RA  model  relative  to  the  LC 
model.  But  combining  the  two  effects  magnifies  the  deterioration. 
If  there  is  high  cost  noise  and  few  observations  on  which  to 
estimate  model  parameters,  the  RA  model  becomes  ver} \ inreliable  and 
the  LC  model  is  strongly  superior. 

Finally,  Figure  11  shows  the  interaction  between  Production 
Rate  Instability  and  Production  Rate  Trend.  These  variables 
reflect  two  aspects  of  production  rate  during  the  model  estimation 
period.  The  plot  illustrates  that  when  there  is  little  trend  in 
rate,  coupled  with  little  variability  in  rate  relative  to  trend, 
then  there  is  little  basis  on  which  to  estimate  a  rate  parameter. 
Hence  the  LC  model  strongly  outperforms  the  RA  model. 

SUMMARY  AND  CONCLUSIONS 

The  central  research  question  addressed  in  the  study  was: 
under  what  conditions  does  consideration  of  production  rate  and 
incorporation  of  a  rate  variable  into  a  learning  curve  analysis  of 
cost  lead  to  a  more  accurate  prediction  of  future  cost?  The 
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FIGURE  8 


INTERACTION  OF  PRODUCTION  RATE  TREND 
WITH  FUTURE  PRODUCTION  LEVEL 
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INTERACTION  OF  COST  NOISE 
WITH  DATA  HISTORY 
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INTERACTION  OF  PRODUCTION  RATE  INSTABILITY 
WITH  PRODUCTION  RATE  TREND 
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analysis  simulated  prediction  for  both  a  traditional  learning  curve 
and  a  rate  adjustment  model  and  compared  prediction  accuracy  under 
various  conditions.  One  central  finding  was  that  neither  a 
traditional  learning  curve  analysis  nor  an  analysis  which  considers 
the  impact  of  production  rate  on  cost  is  inherently  superior. 
Neither  models  dominates;  both  outperform  the  other  under  some 
conditions. 

General  tendencies  were  evident.  Considering  production  rate 
in  the  analysis  lead  to  reduction  in  prediction  error  and  improved 
accuracy  when 

The  number  of  observations  available  for  the  analysis 
was  relatively  rich. 

The  amount  of  fixed  cost  in  total  cost  was  relatively 
high. 

The  production  rate  trend  had  been  growing  during  the 
model  estimation  period. 

—  The  period-to-period  variability  in  production  rate  was 
relatively  large. 

—  Random  noise  in  cost  due  to  unsystematic  factors 
impacting  cost  was  relatively  low. 

Production  volume  was  expected  to  be  cutback  in  the 
future  periods  for  which  cost  predictions  were  being 
made . 

Each  of  these  findings  suggests  that  researchers  or  cost  analysts, 
engaged  in  a  cost  prediction  or  cost  analysis  problem,  may  benefit 
from  attending  to  such  factors  when  deciding  on  the  form  of  model 
or  analysis  they  might  bring  to  bear. 

Numerous  interacting  impacts  of  combinations  of  factors  on 
prediction  accuracy  were  also  evident,  but  a  broad  "theme"  was 
apparent  in  the  interactions,  suggesting  a  general  conclusion; 
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—  The  greatest  impact  (of  changes  in  the  various  factors) 
on  relative  prediction  accuracy  (of  the  learning  curve 
approach  and  the  rate  adjustment  approach)  occurs  when 
cutbacks  in  future  production  are  anticipated. 

This  means  that  researchers  and  cost  analysts,  attempting  to 

predict  future  cost  in  an  environment  where  future  production 

volume  is  declining,  will  find  the  choice  of  an  approach  to  be  most 

critical .  The  relative  accuracy  of  the  learning  curve  approach  or 

the  rate  adjustment  approach  is  particularly  sensitive  to  changes 

in  data  richness,  fixed  cost  burden,  production  rate  trend  and 

stability,  and  cost  noise  when  cutbacks  are  anticipated. 

Other  interactions  collectively  suggest  a  second  general 
conclusion: 

—  The  impacts  of  factors  on  the  relative  prediction 
accuracy  of  the  two  approaches  tend  to  be  additive. 

This  means  that  if  the  presence  of  one  factor  (say,  lean  data 

history)  and  the  presence  of  another  (say,  high  cost  noise)  both 

reduce  the  relative  accuracy  of  the  rate  adjustment  approach,  then 

the  presence  of  both  will  magnify  the  effect. 

The  conclusions  of  any  study  must  be  tempered  by  any 
limitations.  The  most  prominent  limitation  of  this  study  is  the 
use  of  simulated  data.  Use  of  the  simulation  methodology  was 
justified  by  the  need  to  create  a  wide  range  of  treatments  and 
maintain  control  over  extraneous  influences.  This  limitation 
suggests  some  directions  for  future  research. 

—  Re-analyze  the  research  question  while  altering  aspects 
of  the  simulation  methodology.  For  example,  are 
findings  sensitive  to  the  cost  function  assumed? 

—  Address  the  same  research  question  using  actual  cost 
and  production  rate  data.  Are  the  same  findings 
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evident  when  using  "real-world"  data? 

Providing  confirmation  of  the  findings  by  tests  using  alternative 
approaches  would  be  beneficial. 

Additional  future  research  may  be  directed  toward  new,  but 
related,  research  questions. 

—  Investigate  other  qualities  of  predictive  performance, 
for  example,  bias.  Do  learning  curves  or  rate 
adjustment  models  systematically  provide  predictions 
that  are  biased  toward  under  or  over  estimation  of 
future  cost?  Under  what  conditions? 

—  Investigate  the  magnitude  of  prediction  errors  using 
either  a  learning  curve  or  rate  adjustment  approach. 
How  large  are  average  prediction  errors  under  varying 
circumstances? 

Investigate  other  competing  models  or  approaches  to 
cost  prediction.  Perhaps  accuracy  can  be  improved  by 
using  some  version  of  a  "moving  average"  prediction 
model.  Can  such  a  model  outperform  both  the  learning 
curve  and  the  rate  adjustment  approach?  If  so,  under 
what  circumstances? 
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